Validating Rover Image Prioritizations

Planetary exploration missions often have the capability of collecting more data volume than the mission is allocated in downlink bandwidth. Currently, instruments often collect and transmit data at a rate that equals their bit limit, remaining idle much of the remaining time. One scenario to take advantage of instrument capability is to permit continuous data collection and to autonomously select onboard the most scientifically valuable images for downlink. Given the same bandwidth constraint, this approach should increase the diversity and value of the transmitted data. A critical issue with this approach is how to prioritize the data. In this article, we present a quantitative technique for evaluating prioritizations. We apply the technique to assess the agreement of human experts on image rankings. These results can provide an upper limit to the kind of agreement we may expect between a set of rankings generated by humans and those produced by an automated algorithm. In addition, our analysis of the results leads to insights for maximizing the performance of automated algorithms that could be used onboard a spacecraft.